A framework for post-project evaluation of multicriteria decision aiding processes from the stakeholders’ perspective : design and application

Numerous multicriteria decision aiding (MCDA) methods have been developed over the last decades and are now applied in various domains, sometimes using facilitated group workshops to create models. These models are all designed to improve decision processes. However, the lack of follow-up and post-project evaluations limit the understanding of how the participants experienced the group workshops and how the results were subsequently used within the organization. This is in contrast with the public participation research field, where a rich literature was developed for a posteriori evaluation of projects. Based on this literature, our research proposes a framework to evaluate, ex-post, MCDA projects. In order to illustrate this framework, we apply it to an MCDA project in Quebec City where a spatial decision support system to prioritize the redesign of streets as Complete Streets was built. Individual interviews were conducted with the Quebec City professionals that currently use, were leaders of the project, or have participated in the development of the decision support system. This research has identified that the need for change of practices within the workplace, communication problems, and the requirement for multidisciplinary work were at the root of the various challenges encountered during the workshops. Based on our experience, we propose some lessons learned and potential solutions that can enhance the body of literature in MCDA

A review of cost–benefit analysis and multicriteria decision analysis from the perspective of sustainable transport in project evaluation

Transport decision processes have traditionally applied cost-benefit analysis (CBA) with benefits mainly relating to time savings, and costs relating to infrastructure and maintenance costs. However, a shift toward more sustainable practices was initiated over the last decades to remedy the many negative impacts of automobility. As a result, decision processes related to transport projects have become more complex due to the multidimensional aspects and to the variety of stakeholders involved, often with conflicting points of view. To support rigorous decision making, multicriteria decision analysis (MCDA) is, in addition to CBA, often used by governments and cities. However, there is still no consensus in the transport field regarding a preferred method that can integrate sustainability principles. This paper presents a descriptive literature review related to MCDA and CBA in the field of transport. Among the 66 considered papers, we identified the perceived strengths and weaknesses of CBA and MCDA, the different ways to combine them and the ability of each method to support sustainable transport decision processes. We further analysed the results based on four types of rationality (objectivist, conformist, adjustive and reflexive). Our results show that both methods can help improve the decision processes and that, depending on the rationality adopted, the perceived strengths and weaknesses of MCDA and CBA can vary. Nonetheless, we observe that by adopting a more global and holistic perspective and by facilitating the inclusion of a participative process, MCDA, or a combination of both methods, emerge as the more promising appraisal methods for sustainable transport

Intégrer le transport durable dans les processus décisionnels pour le réaménagement de rues : application de l’aide multicritère à la décision

Les nombreux impacts négatifs liés à la démocratisation de l'automobile dans le dernier siècle ont mené à repenser les systèmes de transport et les pratiques de planification des transports. Les systèmes de transport devraient être planifiés pour protéger la viabilité écologique à long terme, pour fournir une accessibilité de base et pour assurer un accès équitable intergénérationnel et intragénérationnel au transport : c'est le transport durable. Plusieurs actions et stratégies ont été proposées dans les dernières décennies pour mettre en œuvre le transport durable. Le réaménagement des rues en faveur du transport actif et en commun est une de ces stratégies. Toutefois, il existe actuellement un décalage entre les plans, qui adoptent une vision de transport durable, et la pratique, qui réalise encore des projets de transport non-durables. Les différentes échelles de gouvernements requièrent de nouveaux outils d'aide à la décision pour changer les processus décisionnels. L'aide multicritère à la décision (AMCD) est une approche de mise en œuvre par divers processus possibles et proposant diverses méthodes d'analyse qui est de plus en plus discutée dans la littérature en transport afin d'intégrer une perspective holistique dans les processus décisionnels et l'évaluation de projet. Dans cette optique, cette thèse étudie plus particulièrement l'utilisation de l'AMCD pour améliorer les processus décisionnels liés au réaménagement de rues. Tout d'abord, une revue de littérature a été réalisée afin de mieux comprendre les méthodes d'évaluation actuellement utilisées dans les processus décisionnels en transport. Les deux principales méthodes d'évaluation dans le domaine (analyse coût-bénéfice et AMCD) ont été analysées selon leurs forces et leurs faiblesses perçues, les différentes manières de les combiner et leurs capacités à intégrer les principes de développement durable dans les processus décisionnels. Afin de considérer les différentes conceptualisations de l'aide à la décision, les résultats ont été analysés en considérant quatre approches d'aide à la décision basées sur le concept d'agir communicationnel d'Habermas (objectiviste, conformiste, ajustable et réflexive). Ensuite, une méthode pour développer un outil cartographique d'AMCD qui priorise le réaménagement de rues en rues conviviales a été proposée et a été appliquée en collaboration avec la Ville de Québec, Canada. Les rues conviviales sont l'objet d'un mouvement populaire en Amérique du Nord pour l'aménagement de " rues pour tous ". L'outil a été développé lors d'ateliers multidisciplinaires (transport, urbanisme, environnement, infrastructure, etc.) réunissant des professionnels de Québec selon la méthode d'AMCD MACBETH. Le développement de l'outil a été réalisé en cinq phases : (1) la structuration du problème, (2) la construction d'échelles d'attractivité, (3) l'élicitation de taux de substitution, (4) la validation du modèle et (5) la production de cartes de priorité. Forte du succès de la démarche, la Ville de Québec intègre cet outil dans sa stratégie de rues conviviales depuis 2017. Pour faire suite à l'outil développé avec la Ville de Québec, un cadre d'évaluation post-projet dans un contexte d'AMCD a été développé et appliqué à propos du développement et de l'utilisation de l'outil. Le cadre d'évaluation se base sur cinq questions : pourquoi évaluer?, quoi évaluer?, sur quoi est basée l'évaluation?, comment évaluer? et qui est impliqué dans l'évaluation?. L'évaluation a pris la forme d'une série d'entretiens individuels réalisés avec les professionnels impliqués dans le développement et l'utilisation de l'outil cartographique d'AMCD. Les entretiens portaient sur le développement, l'utilisation et la pérennité de l'outil et ont été analysés selon la méthode d'analyse thématique. Enfin, afin d'évaluer le potentiel d'application de la démarche dans d'autres contextes, les pratiques et les perceptions de professionnels quant à la réfection et au réaménagement de rues provenant de 11 municipalités québécoises ont été documentées lors d'ateliers multidisciplinaires de groupe (infrastructure, transport, urbanisme, environnement et géomatique). Les objectifs de ces ateliers étaient (1) de brosser un portrait des pratiques actuelles et (2) d'identifier les enjeux et les défis à développer un outil cartographique d'AMCD dans des contextes municipaux différents. Les ateliers ont été analysés en schématisant les processus décisionnels actuels et en utilisant la méthode de cartographie causale. Une série de lignes directrices a été proposée pour permettre le développement d'outils cartographiques d'AMCD au sein de municipalités. Ainsi, ces lignes directrices visent à faciliter le développement d'un nouveau type d'outils d'aide à la décision pour les municipalités, mais également à permettre d'améliorer les processus décisionnels actuels des municipalités afin de mieux arrimer les visions et les objectifs adoptés dans les politiques et les plans avec les projets réalisés et les pratiques professionnelles.The numerous negative impacts linked to the democratization of cars in the last century led to rethink transportation systems and transportation planning practices. Transportation systems should be planned to safeguard long-term ecological vitality, provide basic accessibility and ensure equal access to transport services. This is sustainable transportation. Many actions and strategies were proposed in the last decades to implement sustainable transportation. Redesigning streets in favor of active and public transportation is one of these strategies. However, there is currently a gap between the plans, that adopt a sustainable transportation vision, and practices, that still realize unsustainable transportation projects. The different government levels require new decision aid tools to change their decision processes. Multicriteria decision aiding (MCDA) is one method that is more and more discussed in the transportation literature to integrate a holistic perspective to decision processes. In this regard, this thesis studies more specifically the use of MCDA to improve decision processes linked to street redesigns. First of all, a descriptive literature review was conducted to better understand the evaluation methods that are currently used in transportation decision processes. The two main evaluation methods in the field (cost-benefit analysis and MCDA) were examined according to their perceived strengths and weaknesses, to the different ways to combine them and to their abilities to integrate sustainable development principles in the decision processes. To take into account the different conceptualizations of decision-aiding, the results were analyzed according to four decision-aiding approaches based on the concept of communicative action from Habermas (objectivist, conformist, adjustive and reflexive). Subsequently, a method to develop a multicriteria spatial decision support system (MC-SDSS) to prioritize the streets to redesign as Complete Streets was proposed and applied in collaboration with the City of Quebec in Canada. Complete streets is a popular movement in North America for sustainable transportation to design " streets for everyone ". The MC-SDSS was developed during multidisciplinary group workshops (transportation, urban planning, environment, infrastructure, urban design and public participation) gathering Quebec City professionals using the MCDA method MACBETH. The development of the MC-SDSS was split in five phases: (1) structuring the problem, (2) constructing attractiveness scales, (3) deriving scaling constants, (4) validating the model and (5) producing priority maps. The process has been successful. Indeed, Quebec City has been using this MC-SDSS in the elaboration of its Complete Streets strategy since 2017. Following the MC-SDSS developed in Quebec City, a post-project evaluation framework, specific to MCDA, was generated and applied to the MC-SDSS. The evaluation framework is based on five questions: why evaluate?, what to evaluate?, on what is the evaluation based?, how to evaluate? and who is involved in the evaluation?. The evaluation was applied under the form of a series of individual interviews carried with the professionals involved in the development and use of the MC-SDSS. The interviews were about the development, use and future of the MC-SDSS and were analyzed according to the thematic analysis method. Based on the challenges and difficulties identified in the thematic analysis, various recommendations are suggested to improve practices. Finally, to assess the potential to export the development of MC-SDSS to other contexts, the practices and perceptions of professionals from 11 municipalities in the Province of Quebec were documented during multidisciplinary group workshops (infrastructure, transport, urban planning, environment and geomatics). The objectives of these workshops were (1) to create a portrait of current practices for street rehabilitation and redesign to integrate urban planning, transportation and environment and (2) to identify the issues and challenges of developing MC-SDSS in various municipal contexts. The workshops were analyzed by schematizing the current decision processes and by using the causal mapping method. A series of guidelines is proposed to allow the development of MC-SDSS with the municipalities. Those guidelines aim at easing the development of a new type of decision support system for municipalities, but also at allowing the improvement of current municipal decision processes by better integrating the sustainable vision adopted in the politics and plans and the projects realized by the professionals

Assessing and ranking the potential of a street to be redesigned as a complete street: a multi-criteria decision aiding approach

In order to increase the sustainability of transportation in cities, streets must provide infrastructures that favor active and public transportation modes. However, the current decision-making processes in transportation projects do not always explicitly take into account sustainable transportation requirements and often ignore concerns that may arise in other fields such as environmental protection, health, or urban design. The proposed framework in this paper aims at integrating sustainability in a decision-making process by creating better collaboration and communication between professionals of such disparate fields. The framework is based on group workshops, problem structuring, multi-criteria decision aiding and geographic information systems. It was applied in Quebec City, Canada. The objective of the workshops was to identify higher priority streets that should be redesigned as Complete Streets, a popular movement that advocates “streets for everyone” in North America. To facilitate communication and use for future decisions, the results of the analysis were integrated and presented in a geographic information system. As a result, this user-friendly tool is currently used to support decision-making by Quebec City officials who must choose the streets to be redesigned as Complete Streets while ensuring transparency and traceability. Such an approach may better support policies to make the transport system more sustainable by integrating various considerations, being transparent and improving communication of the outcome